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EXECUTIVE SUMMARY 


The employment of unmanned aerial vehicles (UAVs) by the Israeli Defense 
Forces during the Peace for Galilee campaign in 1982, and by United States forces during 
Operations Desert Shield and Desert Storm in 1990-91, provides a proof of concept that 
relatively low cost UAVs can provide effective surveillance, reconnaissance, target 
acquisition and fire support adjustment missions. The United States has employed UAVs 
subsequently in operations in Somalia, Haiti, Bosnia, and most recently in Kosovo. The 
Department of the Navy’s doctrine of Operational Maneuver from the Sea will place 
greater reliance on aerial battlefield surveillance from "over the horizon" or expeditionary 
sites ashore in order to control a greater Operational area with less force concentration. 
However, the current Naval UAV platform, the Pioneer, is beset by an unacceptable 
mishap rate. Since its fielding in operational units in 1986, the Pioneer Class A flight 
mishap rate is 385 mishaps per 100,000 flight hours. This stands in stark contrast to that 
of manned Naval Aviation where the rate is approximately two Class A flight mishaps 
per 100,000 flight hours. 

Schmidt & Parker (1995), while working at the Naval Safety Center in Norfolk, 
Virginia, have identified that human factors related issues cause half of the Naval UAV 
mishaps. Seagle (1997) applies the Human Factors Analysis and Classification System 
(HFACS) taxonomy to UAV mishap reports in order to improve human factors mishap 
investigation, reporting and analysis. The HFACS taxonomy, based upon the Reason 
(1990) "Swiss Cheese" model of accident causation, captures the latent conditions that 


"set the stage” for active failures that lead to mishaps. 
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This study refines the coding of UAV mishaps in accordance with the HFACS 
taxonomy. Once the mishaps are parsed, the factors influencing the mishap rate and their 
resultant costs are computed. These results are used to conduct a stochastic model 
simulation of annual UAV flight operations in order to isolate those categories of mishaps 
that contribute most to increased budgetary costs and decreased mission readiness. The 
study places UAV mishap reports from FY86 to FY98 into categories using the HFACS 
taxonomy. Particular emphasis is placed on the FY93 to FY98 period when Naval UAVs 
first came under the cognizance of the Naval Aviation Safety Program. Then, mishap 
rates and probabilities are estimated for the various categories. Thereupon, these 
probability distributions are used as the input to a stochastic model for the simulation of 
annual flight operations. The model output is the annual mishap costs associated with a 
particular category, and a resulting mission readiness index. 

Intervention strategies are assigned to associated category of mishap causation. 
Comparing the annual mishap cost, readiness index, and the feasibility associated with a 
particular intervention strategy, Fleet users and program managers can determine what 
intervention strategies are mOSt appropriate. Some Strategies are specific to the Pioneer 
system, for example, an engine remanufacturing or upgrading, or an electronic 
weatherproofing modification. Other strategies, such as improved aircrew coordination 
training and personnel assignment stabilization, transcend the Pioneer system, and are 
applicable to all follow-on UAV configurations. 

The mishap categorization phase of this study, data coding and classification, was 


conducted independently of previous studies (Schmidt & Parker, 1995; Seagle, 1997). 
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The relative frequency of mishap causes agrees with the previous classifications, and 
validates their findings. Even though mishap reports remain vague, and lack granularity 
at identifying lower, root causes, there is general agreement in the classification of the 
mishap in each independent analysis. 

The second phase of the study, stochastic model simulation of annual flight 
operations, finds that electro-mechanical causes have a low impact on mission readiness 
although they account for approximately one-quarter of UAV mishaps. On the other 
hand, unsafe acts, unsafe conditions for flight operations, and unsafe supervision have a 
more significant impact on equipment repair and replacement costs, and mission 
readiness. The study concludes that addressing the human factors related issues through 
increased aircrew coordination training, the use of simulators for mission rehearsal, 
personnel stabilization, and the development of a UAV career path, will have a greater 
impact on controlling cost and improving readiness. The study also generates the 
budgetary costs of selected mishap categories. These costs can be compared to the cost of 
implementing a particular intervention strategy. Thereupon, the program manager can 
chose the most appropriate strategies for implementation. While in isolation, no single 
intervention strategy will eliminate mishap occurrences; their implementation will 


increase the density of the "safety net" surrounding UAV operations. 
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I. INTRODUCTION 


A. OVERVIEW 


From the advent of modern military operations through the present day, successful 
commanders have relied on knowledge of the battlefield and intelligence of the enemy’s 
disposition in order to employ their force to achieve victory (Marine Aviation Weapons 
and Tactics Squadron One [MAWTS-1], 1997). During the 19" century, in the American 
Civil War and the Franco-Prussian War, hot air balloons were employed in order to get a 
bird’s eye view of the battlefield. They added a third dimension to the intelligence 
collecting effort. Early in the 20" century, the invention of the airplane replaced the 
restrictive and vulnerable use of balloons. During the First World War, manned aircraft 
were employed as airborne intelligence collectors. Later, in the 1960’s in response to 
increasingly lethal air defense networks aa enormously developing technology, 
reconnaissance satellites have taken over the role of aerial surveillance at the strategic and 
operational levels of warfare. Presently, Unmanned Aerial Vehicles (UAVs) are used to 
augment satellite systems by providing near real-time, tactical aerial reconnaissance. 
However, the current Naval UAV, the Pioneer, is beset by an unacceptable mishap rate. 
This thesis addresses several potential intervention strategies designed to mitigate these 


mishaps, lower their budgetary cost and improve operational readiness. 


B. AERIAL RECONNAISSANCE 


The U-2 "spy plane" was developed during the early 1950s in order to monitor 
Soviet ICBM development and deployment (Jones, 1997). The U.S. conducted U-2 over 


] 


flights of the Soviet Union since 1955. On 2 May 1960, Francis Gary Powers was shot 
down in a U-2 over the Soviet Union by an SA-2 "Guideline" missile. The shoot down of 
Powers, and the failed attempt at covering up the mission, proved to be a devastating 
blow to the U.S.’s international prestige. A second U-2 was shot down over Cuba during 
the Missile Crisis on 27 October 1962, while attempting to determine the status of Soviet 
nuclear missiles. Consequently, the nation became increasingly wary of manned 
reconnaissance. However, the only reconnaissance systems in development were the SR- 
71 Blackbird and the CORONA spy-satellite. Both of these systems were designed for 
strategic level reconnaissance only. 

The U.S. made an initial serious attempt at an airborne tactical reconnaissance 
UAV during the Vietnam War (Jones, 1997). The U.S. Air Force experimented with 
launching the Teledyne-Ryan developed Lightning Bug UAV from MC-130's in order to 
conduct aerial reconnaissance. By the end of the war, the Lightning Bug UAV had grown 
into a vehicle for providing real-time video, electronic intelligence (ELINT) collection, 
electronic countermeasures (ECM), communications intelligence (COMINT) and 
psychological operations (PS YOPS) leaflet dropping. However interest in UAVs waned 
as the war wound down, and the Department of Defense (DoD) trimmed budgets and 
force structure. 

Nevertheless, U.S. lessons learned in Vietnam did not go unheeded. During that 
same period, Israel Aircraft Industries (IAI) began the successful development UAVs for 
the Israeli Defense Forces (IDF) (Kumar, 1997). These Israeli developed and 


manufactured UAVs were first employed in combat in 1982 during the Peace for Galilee 


campaign in Lebanon. During an attack against Syrian forces in the Bekaa Valley, the 
Israeli Air Force (IAF) launched decoy missiles to stimulate the Syrian air defense 
system. As the Syrians responded to this perceived attack, Israeli Mastiff and Scout 
UAVs with electro-optical and radar-detecting payloads were able to locate and target 
Syrian missile sites. Once target locations were confirmed, and during the Syrian missile 
reload cycle, the IAF launched A-4, F-4 and KFIR aircraft to attack these targets. The 
Israelis destroyed 19 SAM batteries and 86 MiG aircraft while only losing one aircraft of 
their own, effectively dismembering the entire Syrian air defense network. 

The early 1980s saw a resurgence in enthusiasm for UAVs within DoD as a result 
of the Israeli’s successful UAV employment in Lebanon. U.S. peacekeeping operations in 
Beirut (1982-3), Operation Urgent Fury (Grenada, 1983), Operation Eldorado Canyon 
(Libya, 1986) all highlighted a requirement for an inexpensive, over the horizon, 
unmanned reconnaissance capability for the on-scene tactical commander. In July 1985, 
the Secretary of the Navy, John Lehman directed the expeditious acquisition of UAVs for 
fleet operations. By December 1985, the U.S. Navy procured the Pioneer UAV system 
developed by IAI. The Pioneer UAV is the next generation UAV following the Mastiff 
that was employed in the Peace for Galilee campaign. The U.S. Navy and Marine Corps 
began establishing UAV units in 1986. The first Pioneer overseas deployment occurred 
in December 1986, aboard the U.S.S. Jowa (Pioneer UAV Inc., 1999). 

The first time the Pioneer UAV saw combat action was in Operations Desert 
Shield and Desert Storm (Melson, Englander & Dawson, 1992). During that conflict, six 


UAV units were deployed to the Kuwaiti Theater of Operations (KTO) - two Navy, three 


Marine Corps and one Army. Together, the units flew 336 missions, accruing 985 flight 
hours. Only three UAVs were hit by enemy anti-aircraft artillery (AAA), resulting in the 
loss of only one UAV from enemy action. General Walter Boomer, USMC (ret.) who 
commanded the Marine Expeditionary Force (MEF) in the Kuwaiti theater summed up 
the effect of UAV employment during the war when he said, "The Pioneer UAV was the 
most significant intelligence collection source within I MEF." 

Pioneer UAV units were later deployed for Operations Restore and Continue 
Hope in Somalia, Operation Uphold Democracy in Haiti, and Operation Joint Endeavor 
in Bosnia. Their successful mission performance reinforced the potential of exploiting 
this technology to support future combat operations and operations other than war 
(OOTW) (Jenkins, 1998). The Department of the Navy’s doctrine of "Operational 
Maneuver from the Sea" (OMFTS) will place greater reliance on battlefield surveillance 
from "over the horizon" and expeditionary ee ashore in order to control a greater 
operational area with less force concentration. Relatively inexpensive reconnaissance 
UAVs with the ability to conduct missions in hostile airspace, without putting pilots into 
harm’s way is a key element of OMFTS (Marine Corps Combat Development Command 


[MCCDC], 1999). 


cc: THE PIONEER UAV SYSTEM 


A Pioneer UAV system consists of five Pioneer air vehicles, a ground control 
station (GCS), a portable control station (PCS), a tracking communications unit (TCU), a 
data link, two remote receiver stations (RRS) and a reconnaissance payload. The system 


can be operated aboard specially configured U.S.S. Austin Class Landing Platform Dock 
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(LPD-4) ships or from prepared airstrips ashore. While UAVs do not carry any ordnance, 
they can perform as forward observers of indirect fire support assets — offensive attack 
aviation, artillery and naval surface fire support. Figure 1 is a representation of how 


UAVs can be deployed during an amphibious operation. 
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FIGURE 1: PIONEER UAV CONCEPT OF OPERATIONS 


Within DoD, only the U.S. Navy, Marine Corps and the Defense UAV Training 
Command (DUTC) operate the Pioneer UAV (MAWTS-1, 1997). The Navy's UAV unit 
is VC-6, headquartered in Norfolk, Virginia; however, the UAV elements are located at 
Webster Field, Patuxet River, Maryland. From there, they deploy as detachments with a 
complete Pioneer system aboard ship. There are two Marine Corps UAV units. VMU-1 
is located in Twenty-Nine Palms, California, and WMU-2 is headquartered at MCAS 
Cherry Point, North Carolina. The training command, DUTC, is located in Fort 


Huachuca, Arizona. The U.S. Army deactivated their Pioneer UAV units in anticipation 


of procuring the Hunter (IAI) or Outrider (Alliant Techsystems) UAV system. However, 
both programs were cancelled because they failed to meet desired size, weight and 
corrosion control specifications (Sherman, 1998). 
Pioneer UAV units are currently tasked to provide the following missions: 
a) Reconnaissance, surveillance and target acquisition (RSTA). 
b) Adjusting indirect fire (Artillery, Naval Surface Fire Support). 
c) Collect Battle Damage Assessment (BDA). 
d) Support security operations (e.g., convoy —— monitor enemy 
avenues of approach and named areas of interest (NAIs). 

The term "unmanned" is a misnomer when applied to the Unmanned Aerial 
Vehicle system because UAV operations involve many remote participants. The essential 
members of a UAV crew include a Mission Commander, an Internal Pilot, a Payload 
Operator and an External Pilot. Additionally launch and recovery teams and maintenance 
personnel will be involved in flight operations (Joint UAV Training Operations 
Procedures & Standardization [JUAVTOPS], 1997). A detailed description of the 


Pioneer UAV system, capabilities and crew responsibilities is attached as Appendix A. 


D. PIONEER UAV SAFETY RECORD 


Schmidt & Parker (1995), of the Naval Safety Center (NSC), begin the initial 
effort at improving UAV operational readiness by soditSing on mishap prevention. Their 
study examines the 107 mishaps that occurred between 1986 and 1993. The breakdown 
of causal factors is illustrated in Figure 2. This research indicates a significant number 


(approximately 59%) of mishaps occur as a result of electromechanical problems. 
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Human error accounts for nearly one-third of these mishaps. Their research indicates the 
following factors present significant safety concerns: crew selection and _ training; 
aeromedical readiness: pilot proficiency/currency; personnel shortages; operational tempo 


(OPTEMPO); human error in teamwork and aircraft control. 
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FIGURE 2: FY 86 — FY 94 NAVAL PIONEER UAV MISHAP CAUSAL FACTORS (SCHMIDT & PARKER, 1995) 


Schmidt & Parker (1995) recommend the following corrective actions be taken to 
decrease the mishap rate: aeromedical screening and monitoring; criteria based selection 
process; UAV crew coordination training; take off, landing and external pilot (EP) 
simulators; inclusion into OPNAVINST 3710.7P NATOPS oversight and improved 
human systems integration. These recommendations have been implemented with 
varying degrees of success. Aeromedical screening is conducted prior to being assigned 
to the UAV community. In addition, aptitude testing is conducted to assess the flight 
potential of incoming aircrew. Aircrew coordination training (ACT) is currently in 


progress and constantly evolving. A take off and landing drone was developed to enable 


EPs to fly an air vehicle without having the entire system activated. In FY 94, UAV 
flight operations were included in the Naval Aviation Safety Program (OPNAVINST 
3750.6Q). 

These improvements have marginally decreased the mishap rate. However, 
improvement is still required. Initially, aeromedical screening takes place but a flight 
surgeon is not assigned to UAV units to continue aeromedical education and crew 
monitoring. The lack of operational experience and personnel stability in UAV units 
limits ACT because lessons learned are difficult to capture and implement. Additionally, 
emergency action simulation drills are not possible because there is no crew simulator. 
Air vehicle drones, called MiGs by their crews, do not have the same aerodynamic 
characteristics of the Pioneer air vehicle. Thus, while the MiG training is effective in 
training the EPs in basic flight controls and procedures, the discrepancies between its 
response and that of the real Pioneer vehicle may reinforce improper handling and result 
in a negative learning experience. 

Seagle (1997) continues the work of Schmidt & Parker. He applies the Human 
Factors Mishap Classification System (HFACS) to analyze UAV mishaps from 1986 to 
1997. His research studies the 203 mishap investigation reports from 1986 through 1997, 
determining that 88 include human related causal factors. His categorization of mishaps 
is illustrated in Table 1. His work provides insight into the eanse of human factors 
related mishaps and clarifies the broad, generalized "human error" labels. Seagle goes 


further by demonstrating that although the primary cause of an accident may have been 


electro-mechanical in nature, a latent cause was due potentially to human factors, either 


contributing to the mishap, or failing to correct a condition that led to the accident. 
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TABLE 1: FY86 —- FY97 UAV MISHAPS PARSED BY CAUSAL CATEGORY (SEAGLE, 1997) 
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Seagle confirms Schmidt & Parker’s recommendations for a full crew simulator to 
enhance the rehearsal of ACT and flight emergency drills. He also reports a trend in the 
following categories: loss of situational awareness, lack or loss of depth perception, 
visual illusions, self-medication and fatigue. In 1996, the Navy’s Bureau of Medicine 
(BUMED) incorporated medical standards for UAV aircrew to address these factors. 
Seagle also proposes an automatic, “hands off" landing system for runway arrestment and 
embarked net recoveries with an override capability for the EP or IP in the case of 
degraded operations. He attributes the lack of urgency in implementation of these efforts 
to the UAV community having no "champion" at the flag officer level. The relatively 


inexperienced and young officers who serve in the UAV community have not reached (or 


may never reach) the rank where they can affect change, unlike officers in manned 


aviation careers. 


E. RESEARCH OBJECTIVE 


The objective of this study is to further identify the causal factors resulting in the 
unacceptable number of UAV mishaps. Particular emphasis is placed on human factors 
related mishaps. Once identified the study constructs a stochastic model to evaluate 
mishap intervention initiatives with the goal of mishap reduction in terms of cost and 
mission readiness. The results of the model are presented to allow decision-makers to 
focus on specific accident causation categories and to choose the most efficient and 


effective intervention strategies for further development. 


F. STATEMENT OF THE PROBLEM 


Since its fielding in 1986, the Pioneer has accumulated a Class A mishap rate of 
385 mishaps per 100,000 flight hours. When this is compared to the manned aviation 
Class A mishap rate of approximately two mishaps per 100,000 flight hours, one sees the 
Homeric proportions of this unacceptable situation. The excessive UAV mishap rate 
translates into significant budgetary cost, degradation in mission readiness, and a 
perception of unreliability by fleet users and those whom they support. Steps must be 
taken to bridge the gap between conceptual capability and actual performance. The 
bottom line is to achieve a dramatic reduction in the UAV mishap rate to an acceptable 
level, to sustain mission readiness, and to minimize mishap costs. Specific research goals 


include the following: 


1. The classification of UAV mishaps under the current aviation mishap 
taxonomy. Additionally, the identification the human factors characteristics that 
significantly impact the UAV mishap rate. 

2. The development of a stochastic model of UAV mishaps which can be used to 
accurately represent mishap occurrences. 

3. The use of the model to analyze the effects of mishap reduction intervention 
Strategies. 

4. The identification of the impact of a particular intervention strategy on cost 


Savings and mission readiness improvement. 


G. DEFINITIONS 


Mishap. A naval mishap is an unplanned event or series of events directly 
involving naval aircraft, which results in $10,000 or greater cumulative damage to naval 
aircraft or personnel injury. Aviation flight mishaps are divided into one of three 
categories based on the severity of the damage, and the cost incurred by the mishap. The 
definitions governing each class of mishap is given in Naval Publication OPNAVINST 
B50 GO. 

Class A Mishap: A class A mishap occurs when the total amount of damage 
exceeds $1,000,000 or if the air vehicle is destroyed. Because the total cost of a Pioneer 
UAV is approximately $1.1 million, a class A mishap will only occur if the UAV is 
damaged beyond repair or lost. Class A mishaps include being lost at sea, or destruction 
of the air vehicle if it forcefully impacts terrain. Loss of a UAV during a combat mission 


is not classified as a class A mishap, but rather, a combat loss. 
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Class B Mishap: This category is used when the total cost of damage is at least 
$200,000 but less than $1,000,000. Usually, a UAV class B mishap occurs when there is 
serious damage to the air vehicle, or if the payload is damaged. The cost of the 
surveillance camera alone ranges from $300,000 to $800,000 depending on whether it is 
the day EO or night FLIR camera. The variance of the cost depends upon whether the 
payload is repairable or permanently damaged. 

Class C Mishap: This category is used when the total cost of damage is at least 
$10,000 but less than $200,000. For UAV mishaps, this situation occurs when there is a 
small amount of damage to the air vehicle possibly from a hard landing, or striking 
another object. Usually, swapping out or remanufacturing parts can repair these types of 


mishaps, keeping costs low. 


H. SCOPE AND LIMITATIONS 


The scope of this research is limited to the Defense UAV Training Command, and 
U.S. Navy and Marine Corps fleet Pioneer UAV squadrons. However, the results of this 
analysis will be pertinent to all UAV operations of current and future systems that 
integrate a human component to conduct the mission. Detailed mishap analysis is 
conducted from FY93 through FY98. In FY93, UAVs were incorporated into the Naval 
Aviation Safety system, and mishap reporting procedures were standardized. Prior to 
FY93, UAV mishap reports contained limited descriptive information, rendering detailed 
analysis nearly impossible. Mishap causes were typically described as pilot error, 


electrical or mechanical failure with little amplifying information. 
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The contents of the thesis are presented in the following order: Chapter Two is a 
literature review of three human error causation models. They serve to provide 
background to the HFACS model used by the NSC, which in turn is discussed in detail 
with specific examples of UAV applications. Chapter Two also discusses accident 
investigations and analysis, mishap intervention strategies, and stochastic modeling. 
Chapter Three describes the methods used to estimate statistical parameters, which are 
inputs to the stochastic model. Further, the simulation methodology is developed. 
Chapter Four includes the mishap database construction, parameter estimates and the 
Output of the stochastic model simulation. Conclusions and recommended courses of 


action are presented in Chapter Five. 
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Hf. LITERATURE REVIEW 


A. ACCIDENT CAUSATION THEORIES 


The Asia-Pacific Safety Magazine (March, 1995), published by the Australian 
Bureau of Air Safety Investigations, reports "between 70% and 80% of [mishap] 
occurrences contain a human factors element." The article continues, "Indeed it can be 
argued that human factors are involved in all occurrences." In a system designed, built, 
maintained and operated by humans, only a fraction of one percent of the mishaps can be 
attributed to factors beyond human control (Bruggink, 1996). 

Because of society's obvious goal peace the number of aviation mishaps and 
eventually their overall prevention, much research has been dedicated to determining 
accident causation, and developing a safe environment where the probability of having an 
accident 1s minimized (Harle, 1993). Nevertheless, the results of aviation accident 
investigations often limit conclusions to phrases such as "pilot error," "failure to see and 
avoid," "improper use of controls," or "failure to observe and adhere to established 
standard operating procedures (SOPs). 

"However, effective accident intervention and prevention requires more than 
identifying who is culpable. In order to implement a safety conscious program, a better 
understanding of the context in which these individuals faced accident conducive 
circumstances is required. 

Within the last thirty years, three accident causation theories stand out for their 


merits in dissecting the myriad of contributing causal factors that create the context of an 
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incident. These three theories are Edward’s (1977) "SHEL" model: Helmreich’s (1990) 
model: and the Reason’s (1990) "Swiss Cheese" model. Each of the three theories 
attempts to explore the latent or removed factors that influence the mishap. The main 
focus is on the "chain of events” and surrounding circumstances that lead to an accident. 


I. The SHEL Model 


The "SHEL" (Software, Hardware, Environment and Liveware) model is first 
introduced by Edwards (1972) and later modified by Hawkins (1984). The SHEL model 
places emphasis on the human being and its interfaces with the other components of the 
man-machine-environment system. Each component of the SHEL model represents the 


components of a modern technological system as depicted in Figure 3. 


Environment Liveware 





FIGURE 3: THE SHEL MODEL 


The human operator is set at the center of the model, and must interact with each 
of the four external components. The edges of the blocks are not simple or straight, 


indicating that the human must be matched with each component in order to function 


properly. A mismatch between blocks, or an improper fit 1s a potential cause for human 
error. In order to analyze the human factors aspects of an environment, one must look at 
both the individual component blocks, and also their interface (Harle, 1993). The 


following is a brief description of each component of the model: 


a) Human Operator 


The human operator is the hub of the SHEL model. As such, analysis of 
the individual must incorporate four categories: physical, physiological, psychological, 
and psychosocial. Physically, one must determine if the individual is capable of 
performing the required task, and if there are any impediments or limitations to successful 
task performance. Physiologically, an individual must be prepared to conduct the task. 
This category includes the items of proper nutrition, alcohol or drug use, tobacco use, 
stress and fatigue, and the effect these have on an individual or crew’s ability to perform 
and make appropriate decisions. Psychologically, an individual must be capable of 
mentally executing the task. Knowledge of what is required, and the confidence to 
perform the task must be established. Moreover, the workload must be appropriate to an 
individual’s information processing and attention capabilities. Finally, psychosocial 
factors impact human performance. Stress, pressure from a supervisor, the workplace 
climate and personal issues all influence one’s reaction in a potentially dangerous 


situation. 


b) Liveware 


The liveware interface 1s the operator’s relationship to the other individuals 


in the workplace. In aviation, this 1s often referred to as crew resource management 
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(CRM) and is addressed later. Liveware also relates to teamwork, morale and the overall 
command climate. Obviously, healthy interpersonal relationships among a crew, or 
between individuals and their supervisors are essential for a safe and effective work 


environment. 


Cc) Hardware 


The hardware interface addresses the human machine interface (HMI). 
The HMI includes workspace configuration, displays, controls, seat design and 
configuration, visibility and climatic conditions. The physical work environment impacts 
crew orientation, information processing, cognition and execution of the task. Similarities 
of component design and their physical layout can affect effective scan patterns, and 
facilitate correlation of input data. The hardware interface is the focus of ergonomic and 


anthropometrical study. 


d) Software 


The software interface is the relationship between the individual and all 
supporting systems found in the workplace. This category includes not only computer 
software design, but also regulations, manuals, checklists, and SOPs. These items must 
be user friendly and understandable to the human operator. Automation is also a 
significant contributor to the software human interface. An entirely automated system 
may have the tendency to breed complacency and boredom for an operator resulting in 
decreased vigilance. On the other hand, the lack of automation can cause task saturation 


for an operator in extreme situations, also resulting in an unsafe environment. 


e) Environment 


The environmental interface is the relationship between the operator and 
the internal and external environment. The internal environment includes temperature, 
lighting, noise, vibration and air quality. The external environment includes visibility, 
weather and terrain. For military applications, the external environment includes the 
combat situation during which operations must be executed. The surrounding 
environment has significant impact on individual and crew motivation, attention, 
judgment and performance. 


2. The Helmreich Model 


Helmreich advised the International Civil Aviation Organization (ICAO) 
Commission of Inquiry on the human factors aspects of an Air Ontario flight accident that 
took place in Dryden, Ontario in 1990 (Zotov, 1996). The advantage of his theory is that 
it eliminates an opportunity for regulators and organizations to argue that their actions 
should not be discussed in the accident report. It is frequently argued that an accident is 
the end result of a chain of events: if a causal factor can be removed and the accident 
could still have occurred, then ipso facto that factor can not be causal. The Helmreich 
model illustrates that an accident is the accumulation of factors, rather than their chaining 
together, which affects a crew’s performance. 

The Helmreich model can be envisioned as a series of concentric circles 
surrounding an operator or crew. Each ring potentially influences the crew and may 
cause a degradation of performance. The four levels of influence are the regulatory 


environment, the organizational environment, the physical environment, and the crew 


ie 


environment (see figure 4). Accidents occur as the result actions taken within the context 


of this crew environment. 





FIGURE 4: HELMREICH MODEL OF ACCIDENT CAUSATION 


a) Regulatory Environment 


The regulatory environment is the guidance for conducting operations. 
Within Naval aviation, this includes NATOPS procedures, the Training and Readiness 
(T&R) manual, unit SOPs, operations orders and commanders guidance. These 
regulations guide supervisors on how to conduct mission planning and execution within 
certain limitations. Examples of these regulations are proficiency and currency 


requirements, crew rest and approved flight profiles. 


b) Organizational Environment 


The organizational environment includes crew composition and_ its 
performance. Consistent training, personnel stability, OPTEMPO and _ leadership 


contribute to the organizational environment. On the micro level, the interface between 
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Operations and maintenance personnel, and on the macro level between a unit and is 
higher and adjacent units within the command and control structure greatly influence the 


organizational environment and the niche into which a crew fits. 


c) Physical Environment 


The physical environment is consistent with the internal and external 
environments of the SHEL model. It also includes the physical condition of equipment at 
the time of an incident, and what affect that had on mission performance. The model 
recognizes that the surroundings of an individual or crew have a significant impact on the 


vigilance and responsiveness of the operator. 


d) The Crew Environment 


The crew environment comprises the interpersonal coordination and 
communication within a crew. The model also extends the crew definition to include its 
interfaces with external control, such as air traffic (ATC) and enroute controllers. 
Anyone associated with the conduct of a mission is de facto part of the extended crew. 
The crew environment is also influenced by CRM. Finally, the crew is further broken 
down into individual components. Individuals possess their own performance strengths, 
weaknesses and vulnerabilities. 

The Helmreich model of accident causation is crew centered. However, instead of 
“blaming” the crew or individuals on the crew, it looks at the context of the accident that 
arises from the faulty actions or decisions by external agencies. These contributing 
factors, although potentially missing from the "chain of events" are equally influential in 


an accident occurrence (Zotov, 1996). 
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3. The "Swiss Cheese" Model 


The "Swiss Cheese" model is the outgrowth of research examining human error 
by Reason (1990). The model discusses the layers of defense used to protect against 
accident occurrences. Reason’s model identifies four layers that potentially contribute to 
an accident: organizational influences, unsafe supervision, unsafe conditions and 


individuals performing unsafe acts (see Figure 5). 
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FIGURE 5: REASON’S SWISS CHEESE MODEL 


In an ideal world, the defensive layers would be intact, preventing the "accident 
trajectory," as depicted in the diagram by the arrow, from passing through to the accident 
event. However, each layer has weaknesses and gaps that are revealed by the holes. In 


the real world, these holes are not fixed and static, otherwise they could be identified and 
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repaired. Reason (1998) describes the holes as dynamic and in a constant state of flux. 
Local conditions drive which defensive layer comes in and out of the frame at a particular 
time. 

Reason (1998) attributes the holes as either active failures, or latent conditions. 
Active failures are either violations or errors that occur in the immediate vicinity of the 
accident occurrence. They are performed by the operators — pilots, air traffic controllers, 
police officers, control room operators, maintenance personnel, and so on. The discovery 
and mitigation of this active failure immediately prior to the accident would most likely 
prevent the accident from happening. Twenty years ago, the discovery of this unsafe act 
would have ended an accident investigation. However in today's climate, unsafe acts are 
seen more as consequences than principle causes. It is recognized that people working in 
complex systems make errors or violate procedures for reasons that go beyond an 
individual. These causes are called latent conditions. 

Latent conditions can be present for many years before they combine with local 
circumstances and active failures to penetrate they systems layers of defense. Examples 
of latent conditions include poor design, improper training and supervision, undetected 
manufacturing defects or poor design, unworkable procedures, or improper automation. 
At the macro level, government, regulator or corporate policy shapes the organizational 
culture, creating the error producing factors within an individual environment. 

Latent conditions are present in all systems and are an inevitable part of the 
organizational culture. Latent conditions are not bad policy decisions, but can result from 


the demands of a limited budget or manpower management constraints. These latent 
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conditions can lie dormant for years and have no impact until they become manifested at 
a time where particular weaknesses in the defense become exposed. In contrast to active 
failures, which tend to be unique to a specific event, latent conditions can go 
unrecognized and can contribute to a number of different accidents. Latent conditions 
increase the likelihood of active failures through the creation of local factors allowing 


error and violations to occur (Reason, 1998). 


B. HUMAN FACTORS ANALYSIS AND CLASSIFICATION SYSTEM 


The Human Factors Analysis and Classification System (HFACS) taxonomy as 
developed by Shappell & Wiegmann (1997) at the Naval Safety Center (NSC), is based 
on Reason’s concept of human error in accident causation. HFACS also incorporates, 
albeit to a lesser extent, components of the SHEL and Helmreich model to define the 
context surrounding an accident. The HFACS taxonomy will be added to the next 
publication of OPNAVINST 3750 and become the standardized methodology adopted by 
the Naval Aviation Safety Program for human factors mishap investigation. HFACS 
attempts to capture the context in which an accident occurs by categorized failures into 
four separate tiers depicted in Figure 6. These tiers are organizational influences, unsafe 
supervision, unsafe conditions, and unsafe acts. For the purposes of this discussion, each 


tier is viewed through the context of naval UAV operations. 
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FIGURE 6: HFACS MISHAP CAUSATION CATEGORIES 
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The first HFACS tier of mishap causation is organizational influences. 
Organizational influences are often difficult to quantify and can rarely be tied as a 
specific cause to an accident. However, their existence certainly contributes to the 
circumstances surrounding an accident. The author discussed these influences with UAV 
crewmembers at VMU-1, VMU-2 and VC-6. Among aviators, there is a perception that a 
UAV tour shows a lack of competitiveness with one’s peers who are in a flying billet in 
their primary airframe. Being assigned to the UAV unit relegates them to the status of a 
second-class citizen. In contrast, several see the UAV field as an opportunity to excel ina 
challenging and developing community. As such, they channel their drive and energy to 
guarantee success. 

Another organizational influence is the lack of a UAV specialty and career path. 
Until 1996, UAV enlisted personnel did not have a specific rating or military operational 


specialty (MOS). As a result, personne] were assigned to a UAV unit with no previous 
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background, underwent training and left the unit three years later, never to return again. 
Initial accessions to UAV units were recruited by advertising for individuals whose hobby 
was remote control airplanes. Initially, a unit could be made up of truck drivers, 
mechanics, yeomen, or any combination of backgrounds. The establishment of enlisted 
specialties has improved the situation. Nevertheless, there are no officer specialties or 
career paths. It is still the case that an officer will only serve one tour in a UAV unit 
before leaving the community. Typically, all officers in a UAV unit, including the 
commanding officer, will be new to the community upon assignment (MAWTS-1, 1997). 

A third organizational influence within the UAV community is the perception by 
others in the aviation field. Besides the normal rivalries that exist among communities, 
aviators typically relegate UAV crews to being the lowest on the totem pole. Often, their 
inclusion in aviation planning and operations in regarded as a nuisance by those who do 
not understand the unique integration requirements that UAV operations must address. 
These organizational conditions, unhealthy at times, can create an environment 
susceptible to unsafe operations and potentially lead to mishap causation. 


Zi Unsafe Supervision 


Unsafe supervision is the second HFACS tier that contributes to an actual mishap 
occurrence. Unsafe supervision can be both unforeseen and foreseen. Unforeseen unsafe 
Supervision includes unrecognized unsafe operations, inadequate documentation and 
inadequate design. Unrecognized unsafe operations result from a supervisor not 
recognizing an unsafe act or condition. For example, a supervisor may not be aware of 


accumulated fatigue among aircrew or maintenance personnel in a unit. Instead of taking 
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corrective action, the situation is ignored. Also, by not being aware of influences on 
individuals, crew assignments and personnel management suffer. An example is the 
supervisor who is not aware that a crewmember has a sick spouse or child, a recent death 
in the family, or marital difficulty. As a result, the supervisor does not know to take these 
adverse mental conditions into account when making crew assignments. 

Inadequate documentation refers to unknown "bugs" in the system. Designed for 
combat, the Pioneer was not fully tested until Operation Desert Storm. Operational 
testing and training are designed to closely resemble combat, but cannot replicate it. 
During Operation Desert Storm, the Marine Pioneer units operated as part of a full scale 
Marine Expeditionary Force (MEF). A microwave antenna/transmitter was located at the 
UAV airstrip for communications. The microwave signal caused electro-mechanical 
interference (EMI) with the UAV uplink and downlink causing loss of vehicle control 
resulting in crashes on several occasions. peeause the Pioneer had not previously 
operated in close proximity to this specific communications equipment, this condition 
was neither realized nor documented. 

Inadequate design is an extension of inadequate: documentation in that an 
inadequate condition exists that is unintentional. The engineers who designed the 
equipment may not have anticipated requirements for certain performance characteristics 
or capabilities. Although it may seem inconceivable for a naval UAV, the Pioneer was 
not built to fly through rain or visible moisture. It was designed by the Israeli Aircraft 


Industries (IAI) for operations in arid areas around Israel. It has a laminated wooden 
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propeller that delaminates in moisture, and the electronic components on the air vehicle 
are not waterproof. This inadequate design has been the cause of several UAV mishaps. 

Foreseen unsafe supervision is the mismanagement of individuals at the personal 
level. It includes the lack of or inadequate supervision, the failure to correct a known 
problem, or a supervisory violation. Foreseen unsafe supervision is a lack of leadership 
and guidance in a crew or in the entire unit that creates an unsafe situation for flight 
operations. Within a unit, this can be an underlying condition that is reflected in lack of 
discipline, operational focus or morale. Unsafe supervision is rarely an isolated instance, 
but is symptomatic of underlying conditions, which can all lead to a mishap. 


= Unsafe Conditions 


Reason (1990) addresses the category of unsafe conditions in accident causation, 
however Shappell & Wiegmann’s (1997) HFACS taxonomy further subdivides these 
conditions into aeromedical, crew resource management and readiness violations. The 
following paragraphs characterize each of these subdivisions. Additionally, each 


subdivision is discussed within the context of UAV community issues. 


a) Aeromedical Conditions 


Aeromedical conditions include the physiological and mental condition of 
individuals, and their physical and mental limitations. The physiological condition of an 
individual includes the functioning of their sensory system and physical condition. A 
UAV crewmember may experience spatial disorientation or visual illusions caused by the 
remote controls of the air vehicle. Unlike manned aviation where pilots incorporate 


vestibular inputs with visual cues to perceive the attitude and motion of the aircraft, a 
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UAV operator does not have those inputs. In order to operate effectively as a 
crewmember, there are required sensory thresholds that must be maintained. In addition 
to the senses, other physical conditions can affect the crewmember. Though not 
exhaustive, medications, fatigue, change in circadian rhythm, hypoglycemia, use of 
alcohol, or vitamin deficiencies can cause a person to be physically unqualified to 
conduct a mission (Edwards, 1990). 

The adverse mental condition of a crewmember is also an aeromedical 
condition that can adversely affect mission performance. The effects of stress and mental 
workload on human performance are widely documented. Both can cause a loss of 
situational awareness, mental fatigue and task saturation. Additionally, personality traits 
and attitudes such as overconfidence, complacency, misplaced motivation, or a desire to 
please, cause an adverse mental state. Increased OPTEMPO, and associated family 
separations, financial concerns, operational or combat fatigue and competition among 
members of a crew, all potentially combine to create the adverse mental conditions that 
can cause an accident (Hawkins, 1987). The final aeromedical contribution to unsafe 
conditions is the physical and mental limitations of the crewmember. Individuals must be 
physically and mentally screened for selection as UAV aircrew in order to prevent the 


environment for the previously discussed conditions to occur. 


b) Crew Resource Management 


Jensen (1995) defines crew resource management (CRM) as the effective 
use of all resources (hardware, software, and liveware) to achieve safe and efficient flight 


operations. The resources that a crew manages are people (other crewmembers), 
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equipment (instruments and controls) and other items such as charts, checklists and 
operational manuals. UAV operations, as stated earlier, involve a crew of at least four, 
and normally more crewmembers. The competency and experience of these individuals, 
their supervision, their interpersonal communication and the remote aspect of the air 
vehicle from the operational control site combine to make unique CRM demands on a 
UAV crew. The crew must communicate via an intercom system (ICS) and 
crewmembers often cannot establish eye contact with one another. Additionally, not all 
crewmembers have access to flight instrument information. As the number of 
crewmembers goes up, the potential for conflicting interpretations of information also 


increases, potentially causing confusion and indecisive actions. 


Cc) Readiness 


Readiness violations refer to violations of standard operating procedures 
(SOPs), rules, and instructions designed to provide a safe operating environment for flight 
operations. Among other things, NATOPS defines regulations on crew rest, self- 
medication and alcohol consumption. In the Marine Corps, the Training and Readiness 
(T&R) Manuals (MCO 3500.21: Volumes I and VI), define crew proficiency and 
currency requirements for an individual conducting a specified mission. Violations of 
any of these regulations can create unsafe conditions for a mission. 


4. Unsafe Acts 


The fourth HFACS tier is unsafe acts, which can be classified as either intended 
or unintended. Intended acts are either mistakes or violations, whereas, unintended acts 


are either slips or lapses. The following paragraphs characterize each of these 
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subdivisions. Additionally, each subdivision is discussed within the context of UAV 


community issues. 


a) Intended Acts 


Mistakes are failures to formulate the correct intentions, and can result 
from shortcomings of perception, memory and cognition. In these cases, the intended 
action is wrong. Knowledge based mistakes are caused by failures to understand the 
Situation and arriving at an incorrect course of action. In this situation a UAV operator 
may be saturated with raw information, and may not be able to process the data to 
formulate the correct actions. Also, because of inexperience, an operator does not have 
the capacity to assimilate the information given to formulate a correct response. 

In contrast, someone with more experience who misapplies a rule under 
certain conditions usually makes a rule-based error. A rule-based decision can be likened 
to if...then logic. These mistakes normally occur in one of three ways. The first is that a 
Tule is followed, but exceptions to the situation are not noted. An example is a UAV 
operator who has been operating from a shore location and is now operating at sea. 
Emergency procedures that are learned and reinforced through training ashore may not be 
applicable or desirable at sea. A mistake occurs when the operator applies a land-based 
procedure at sea and an accident occurs. The second type of rule based mistake is when 
the if... part of the situation is misinterpreted, and the ...then action is inappropriately 
applied. And finally, the third type of rule based mistake is when the if... observation is 


correct and the ...then action is incorrectly applied. (Wickens, 1992) 


ol 


In contrast to mistakes, violations are the willful breaking of rules or 
procedures. Violations can be categorized into routine or exceptional violations. A 
routine violation tends to be habitual in nature and is typical of an individual’s behavioral 
repertoire. An example of a UAV related routine violation is when a crewmember 
habitually fails to follow the unit SOP or NATOPS procedures. This can be as harmless 
as failing to brief a certain emergency procedure during a pre-flight brief. Lack of 
supervision and correction action acerbates the violation by passively reinforcing 
incorrect procedures. On the other hand, an exceptional violation is an isolated departure 
from authority, neither typical of the individual nor condoned by supervisors. The 
deliberate decision to ignore directions from an air controller is an example of an 


exceptional violation. 


b) Unintended Actions 


A slip is an unintended error in which the correct intention is incorrectly 
carried out, as opposed to a mistake where the incorrect intention is correctly carried out. 
For example, the internal pilot (IP) uses dials to control the speed and altitude of the air 
vehicle. Intending to adjust the UAV airspeed and inadvertently changing the altitude 
dial is an example of a slip. According to Wickens (1992), slips occur for three reasons: 
(1) the intended action involves a slight departure from the routine, frequently performed 
action; (2) some characteristics of the stimulus Sinenmen or action sequence closely 
relate to the inappropriate, but more frequent action; and (3) the action sequence is 


automated and therefore, not monitored closely by attention. 
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Finally, a lapse is the failure to carry out an action. Memory failure, 
memory overload, or interruption can cause a lapse. Prior to every UAV mission, all 
members of the UAV crew perform various tasks as outlined in a preflight checklist. If 
that sequence of events is interrupted unexpectantly, requiring the crew or an individual 
to divert attention and then come back to the pre-flight sequence, a particular step may be 
skipped. This momentary distraction can cause a lapse to occur with unknown 


consequences. 


Ce ACCIDENT INVESTIGATIONS, ANALYSIS, AND REPORTS 


Bruggink’s (1996) research in civil aviation points out that accident reporting is 
too preoccupied with reactive, formal responses to stated accident causes. Typical 
investigations focus on the black and white elements of accident causes. Since the role of 
human factors is often a gray area, it can seldom be accommodated by the rules of 
evidence favored by investigating authorities. Emphasis is placed upon direct cause- 
effect relationships, and causal statements become official designators of blame. The 
inclusion of contributing factors to causal statements perpetuates a distinction between 
primary causes and contributing factors that has the effect of lessening their relationship 
to the accident. McAdams, a senior and respected member of the National Transportation 
and Safety Board (NTSB) commented after a 1978 mid-air collision investigation that: "A 
contributing factor is not a primary cause; it is more remote and does not carry the same 
weight or implications as that of a probable cause.” (Bruggink, 1996) As a result, the 


context of the situation is dismissed easier than the individual acts themselves. 
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Zotov (1996) also concludes that there exists reluctance to closely analysis human 
factors related causal factors. Often investigations lead to personal attacks, legal 
obstruction and corporate pressure on the investigating authority if the entire "system" is 
under investigation. A corporation will attempt a legal response that frees them from 
culpability by using the "chain of causation" approach to accident occurrences. 
Additionally, the legal concept of "remoteness of damage” can make it difficult to present 
the concept of a complex network of interacting events that caused an accident. 

Mayer & Ellingstad (1992) state that accident databases frequently describe 
attributes of the physical environment and equipment, but that detailed analysis of 
accident causes, including human factors information, are frequently not represented 
because they are too difficult to obtain and code. | Engineers and front-line operators 
design most accident reporting systems with limited backgrounds in human factors. This 
results in a lack of suitability for addressing nace factors issues. Naval Aviation is 
continuing to attempt addressing this shortcoming through the Naval Aviation Safety 
Program. 

The purpose of the Naval Aviation safety program is-to preserve both human and 
material resources in order to enhance operational readiness. In order to accomplish this 
goal, damage and injury must be addressed to mitigate the hazards inherent in flight 
operations (OPNAVINST 3750.6Q, 1997). The use of HFACS in mishap reporting is 
designed to improve the quality of mishap reports, which will in turn improve the data 
available for analysis. With a higher quality database available, mishap analysts can 


isolate recurring causal factors and recommend suitable strategies to eliminate or reduce 
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the frequency of these accidents. These intervention strategies can then be evaluated for 


their effectiveness in improving operational readiness and reducing budgetary losses. 


D. MISHAP INTERVENTION STRATEGIES 


Mishap intervention strategies are designed to address common mishap causes 
and decrease the probability of their occurring. Typically, these strategies fall into one of 
three categories: engineering controls, policy and procedures and individual protection 
measures. Engineering controls deal with a physical reconfiguration of the system. It 
may be ergonomic, or mechanical in nature. The result of the change is improved system 
performance. Policy and procedures address the circumstances surrounding operations. 
Within the HFACS model, these strategies reflect mishaps in the unsafe conditions and 
unsafe supervision. Changes in flight prerequisites and improved time to train are 
examples of policy modifications. Individual protection refers to the equipment a person 
may use to decrease the possibility of physical injury due to operations. This study 
addresses seven mishap intervention strategies. 


De Aeromedical Screening and Education 


Among the recommendations made by Schmidt & Parker (1995) several are 
associated with aeromedical screening and monitoring. Currently, this is taking place 
prior to assignment to UAV training. However once initial training is complete, a flight 
surgeon is not assigned to each UAV unit. There still exists a perception that UAV 
crewmembers are just like any other ground community, where crew rest and other 
requirements are perceived as a luxury and not a requirement. Additionally, ongoing 


training is not routinely conducted to address the affects of nutrition, alcohol and tobacco 
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use, stress and psychological readiness on mission performance. The first intervention 
strategy addressed is continuing and increasing the education and follow up attention to 
aeromedical issues and their effect on readiness. This intervention strategy is a policy and 
procedure change and is modeled by the unsafe condition (UC) causation category. 


a Aircrew Coordination Training / Crew Resource Management 


The policy and procedural recommendation for mishap intervention is increased 
aircrew coordination training (ACT)/CRM. This is being conducted in UAV units; 
however, the lack of experience among crewmembers limits its effectiveness. Because 
UAV operators are cyclically new to the community, there is no senior leadership that can 
speak with a voice of experience to situations that may occur during a flight. Aviators on 
the crew bring a generic ACT background to a unit, but lack specific UAV applications. 
This results in a reactive rather than deliberate approach to an emergency situation. 
Improved, standardized training will impact mishaps caused by unsafe acts (UA) and 
unsafe conditions (UC). 


of UAV Flight Simulator 


A UAV mission simulator can further enhance mission performance skills and 
mitigate unsafe actions during the conduct of a flight. Schmidt & Parker (1995) find that 
59% of mishaps occurred as a result of electrical, mechanical or engine failure. If these 
Situations can be replicated via simulation, a crew can rehearse procedures to correct the 
problem, or put the aircraft in an attitude where damage is minimized. Currently the 
Pioneer does not have a simulation capability where an instructor can induce such an 


emergency, observe the crew response, and then debrief the crew on their performance. 
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The only time the entire crew can operate as a team to address an emergency is during an 
actual flight. A replacement to the GCS is under development as the Tactical Control 
System (TCS). The TCS is designed to be a universal ground control and 
communications shelter that has the ability to interface with all follow-on DoD UAV 
systems. One of the TCS’s requirements is the ability to conduct simulator training 
without conducting an actual mission. The model addresses unit training procedures and 
is modeled in the unsafe act (UA) category. 


4. Automated Take off and Landing System 


Schmidt & Parker attribute 32% of mishaps to take-off and landing error. 
Although not currently a requirement for future UAV systems, several contractors are 
developing an automated take-off and landing system. Their concept is to fly the UAV to 
a predetermined handover point where it flies into a radio beacon. The beacon then takes 
over sending navigational information to the UAV until it is safely on deck. There would 
also be a manual override system. This engineering control will lower the incidents of 
unsafe acts (UA) during the takeoff and landing phase of the flight. 


5: Personnel Stabilization 


In order to lessen the effects of unsafe supervision as well as unsafe acts. The 
UAV community can establish officer specialty codes and career progression 
possibilities. The current situation of constantly rotating first time supervisors into UAV 
units causes instability and limits the "corporate knowledge" of squadron members. If 
implemented, supervisory and performance skills will dramatically improve. The policy 


change is modeled by the unsafe acts (UA) and unsafe supervision (US) categories. 
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6. Engine Replacement 


Although not human factors related, engine failure has been attributed to one 
quarter of UAV mishaps (Schmidt & Parker, 1995). Engineering modifications replacing 
the current engine with a more reliable engine is modeled in the analysis section. 


7 Electronic Waterproofing 


Like engine failures, electronic failures account for another quarter of UAV 
mishaps. Although not originally designed as a naval UAV, the Pioneer is operated by 
the U.S. naval services and would benefit by environmental shielding. The affect of 
engineering modifications such as water proofing system components for operation at sea, 


in the littorals, and in foul weather is addressed. 


E. STOCHASIC MODELING 


Ross (1997) states that in making a mathematical model for a real world 
phenomenon, it is always necessary to make simplifying assumptions so as to make the 
mathematics tractable. However, making too many simplifying assumptions can make 
our conclusions not applicable to the real world. Therefore, the stochastic model must 
strike a balance between simplicity and realism. 

Law & Kelton (1991) state that mathematic model simulation is one of the most 
widely used techniques in operations research. The mathematical model is used to 
represent a system in terms of logical and quantitative relationships that can be 
manipulated to see how the model reacts. A stochastic process is the collection of 
random variables ordered over time, which are defined on a common sample space. A 


stochastic simulation model takes random variable input components and repeats the 
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simulation multiple times in order to achieve a random, although converging solution. 
The output of a stochastic model is itself random, but the number of repetitions can 
decrease the variance in the results. 

In order to create a stochastic simulation model, probability distributions and 
parameters must be identified. In lieu of using the empirical distribution that may contain 
some "irregularities," particularly if the sample size is small, a theoretical, parametric 
distribution is used to smooth out the data (Law & Kelton, 1997). Mintz (1954a) models 
taxi cab accidents in order to determine whether they could be modeled by a specific, 
known distribution. In order to accomplish this simplification, Mintz makes two 
assumptions: (1) accident liability (or proneness) of people is not changed by accidents in 
which they are involved and do not vary over time; and (2) accident liability is distributed 
in some known manner. Through his study of over 1200 taxi cab accidents, Mintz 
(1954b) concludes that accident rates closely We prone a Poisson Process because (1) 
they do display a "memory less" property; and (2) they are distributed as an exponential 
random variable. For the purpose of the UAV mishap model, parameterized distributions 
are tested to determine their suitability as model inputs. — 

The goal of the stochastic model is to probabilistically simulate annual Pioneer 
UAV flight operations for the Navy and Marine Corps to approximate mishap events, 
their cost, and effect on readiness. The model simulates these mishaps as a Poisson 
Process. The model is designed with an open architecture to model any durations of 


flight operations, or quantify other measures of performance (MOPs). While it may not 
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be feasible or desirable to modify the Pioneer UAV with these results, they are applicable 
for addressing the next generation replacement system. 

In order to model intervention strategies, this model isolates causal factor 
categories. Although, none of these intervention measures can be totally successful when 
employed in isolation within a mishap category. For example, ACT relies on experience, 
which is tied to specific UAV occupational fields and career progression within the 
community. Simulators also enhance training and readiness. Aeromedical readiness is 
tied to standardization and supervision. ll aspects of mishap prevention are 
interdependent. 

Stochastic modeling is used effectively in previous studies to effectively model 
accidents and their effects on cost and missed working hours. Schmorrow (1998) uses 
the HFACS taxonomy to code aviation maintenance mishaps, and stochastic modeling to 
predict cost and readiness. Sciretta (1999) uses a similar methodology to stochastically 
model U.S. Navy shipboard electrical shock mishaps. Teeters (1999), applying HFACS, 
studies the distribution of major and minor aviation maintenance mishaps for Fleet 
Logistic Support (VR) Wing aircraft. This study applies a similar framework to 


stochastically model UAV mishaps. 


ld SUMMARY 


In order to effectively analyze mishap intervention techniques, effective coding 
and documentation is required. The Naval Aviation Safety program provides the 
framework and resources to collect this data.) HFACS is the most recent improvement to 


aid in accident investigation, reporting and analysis. HFACS is based upon accepted 
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accident causation theories. Using HFACS coding, mishaps can be categorized, and 
mishap probabilities determined. To analyze these categories, a stochastic model can be 
used to simulate mishap occurrences over a defined period of time. While random in 
themselves, the results are used to weigh the various options available to reduce mishap 
cost and increase mission readiness. 

Applying the HFACS taxonomy, accidents happen as a result of a confluence of 
weaknesses in all four tiers of the model. Intervention strategies are designed to reinforce 
the cohesion of the tiers and decrease the probability of a window of accident 
opportunity. Intervention strategies are designed to strengthen safety environment at each 


of the four levels. 
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Hil. METHODOLOGY 


A. RESEARCH DATA 


The goal of this research is to use a stochastic model simulation to predict the 
effects of mishap intervention strategies on both operational readiness and budgetary 
costs. The data inputs must be determined in order to conduct the simulation. First, a 
database is constructed from the mishap reports, and causal factors are coded and parsed 
from the data. Analysis of the data leads to the statistical determination of the model 
inputs: inter-event times for mishaps; the mishap rate parameters; the probability of 
mishaps by class (A, B or C); the mishap Pont distribution; and the annual UAV flight 
hours. Once calculated, these statistics are used in a stochastic model simulation of 
annual UAV flight operations. The output of the model is expressed as a mission 
readiness factor, and annual mishap costs. Detailed description of each phase is 
discussed below. 


1. Mishap Records 


The Naval Safety Center (NSC) is responsible for maintaining aviation mishap 
records for the Navy and Marine Corps. The foundation of the mishap record is the 
mishap investigation report (MIR). MIRs are required for all Class A, B and C mishaps 
in accordance with OPNAVINST 3750.6Q. The following items are described in the 
MIR: the events leading up to a mishap, the location and type of operations involved in 
the mishap, causal factors and recommendations for reducing the risk of similar type 


accidents occurring. Prior to FY93, UAV mishaps were not incorporated into the 
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Aviation Safety Program. As a result, the MIRs are not complete and normally point only 
to one causal factor, usually mechanical, electrical or human error. MIRs submitted since 
October 1992 have improved mishap records significantly. Primary and contributing 
factors are reported with greater detail. 


iz Data Base 


The database for this thesis is constructed from the UAV MIRs maintained by the 
NSC, and formatted into an EXCEL spreadsheet. Each mishap event contains the 
following categories: mishap date; air vehicle number; unit; mishap summary; aircraft 
equipment and damage; repair cost; time, location, altitude, and weather at the mishap 
occurrence; causal factors; and recommendations. All repair costs are converted into 
FY98 dollars using the aviation price inflation indices provided by the Naval Center for 
Cost Analysis (NCCA). 

This study addresses mishap rates from the entire database (FY86-FY98). When 
detailed analysis of causal factors is required, it limits its scope to the FY93-FY98 MIRs 
that are standardized by the Aviation Safety Program. Additionally, this partition helps to 
focus analysis on recent steady state UAV operations. Initial mishaps caused by the 
introduction of the air vehicle into the Fleet inventory or by the aberrations to normal 
operations such as those occurring during Operations Desert Shield and Desert Storm do 
not confound the data. 


Se Causal Factors 


A single mishap typically has several codes associated with it. This analysis goes 


beyond the primary causal factor, and addresses known contributing factors. Also of 


note, mishap coding is done at the lowest level possible given the data provided. For 
example, a UAI (unsafe act: intended) is a subset of UA (unsafe act). However, if the 
MIR provides limited information, coding is done at the highest level discernable. The 
UAV mishaps are coded by causal factor in accordance with the Human Factors Mishap 
Classification System (HFACS) (Shappell & Wiegmann, 1997). Material failures are 


recorded by causal factor. Table 2 contains the codes used in the database: 
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TABLE 2: MISHAP DATABASE CAUSAL FACTOR CODES 
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B. DATA ANALYSIS 
ve Inter-event Times 


The computation of mishap inter-event times requires that a transformation be 
made from inter-event days, as recorded by the MIR, to inter-event flight hours, for use 
by the model. In order to accomplish this transformation, annual flight hours are assumed 
to be uniformly distributed throughout the fiscal year. An estimate is made of daily flight 
hours using annual flight hours flown information. This daily flight hour rate is 
multiplied by inter-event days to transform inter-event times from days to flight hours 
flown. 


2. Mean, Variance, and Rate Parameters 


Unbiased estimators for the mean and variance of inter-event times are determined 
for each mishap category that is modeled by the simulation. The mishap rate (A) is 
calculated by taking the inverse of the mean inter-event time (Ross, 1997). The 
Kolmogorov-Smirmov goodness of fit (KS g.o.f.) test is used to decide whether an 
exponential distribution with parameter (A) is an appropriate model for the inter-event 
data, and thus whether the occurrence of mishaps could be modeled as a Poisson Process 
(Conover, 1999). The mishap rate for the entire data set and the mishap rates by causal 
factor are the components of the stochastic counting process, which is being modeled. 


“i Probability of Mishap by Class 


The number of mishaps by class (A, B, C) is recorded for each mishap category. 


The probability of Class A, B and C mishaps is estimated by the number of mishaps in 
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each class divided by the total number of mishaps in that category. A vector of Class A, 
Class B and Class C probabilities is used as a multinomial input to the model simulation. 


4. Cost Distribution 


A cost distribution of each mishap class is determined by estimating the mean and 
variance, and then performing a K-S g.o.f. test to confirm their suitability. The combined 
mishap rates, probability of accident severity and cost distributions are used to describe 
the model in terms of a compound Poisson Process. 


De Determining Annual Flight Hours 


A regression analysis of annual flight hours by fiscal year is used to predict annual 
flight hours for the stochastic model. The model uses the FY99 flight hour prediction as 


the input. The time is the bound in the simulation for the mishap generation period. 


C. SIMULATING THE EFFECTS OF MISHAP INTERVENTION 


The mishap simulation model inputs are the number of simulation repetitions, the 
mishap rate parameter, the multinomial probability vector of the mishap categories (A, B 
and C), and the time period to be simulated. The program instantiates two vectors to 
store the mission readiness factor, and the budgetary cost value. Each simulation run 
goes from time zero to the end time period input value. Time steps are made as a mishap 
is generated using the exponential distribution with the appropriate input rate parameter. 
Once a mishap is encountered, its class is determined randomly, using the input 
probability vector. Mishap cost is calculated using the predetermined cost distribution 


data for that particular mishap class which is "hardwired" into the code. 
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The simulation run ends when the time clock exceeds the input time period. The 
mission readiness factor is calculated by multiplying the number of class A mishaps by 
three, multiplying the number of class B mishaps by two, and adding in the number of 
class C mishaps. If this readiness factor exceeds 21 for the time period, the mission 
readiness is stored as a zero; otherwise, the simulation returns a one — mission ready. 
Both the cost and mission readiness factor are stored in their respective vectors. Once the 
simulation has completed the required number of runs, the simulation returns the mean 
and standard deviation of budgetary cost (FY98$M), and the average of the readiness 
factor. The mishap intervention model code, MishapSim() is programmed in S-Plus 4.0 
and is attached as Appendix B. 

For each mishap intervention strategy, the model is run one thousand repetitions 
and through a total of four simulations. The first simulation is baseline simulation using 
the current mishap rate. A reduction of the mishap rate by 10%, 30% and 50% for each 
intervention strategy is hypothesized for the next three simulation runs. If two or more 
mishap categories are being modeled together, for example, unsafe acts and unsafe 
supervision, the rate parameters are determined as described above. However, all 
replicated mishaps are removed from the composite mishap category. These calculations 
will enable fleet users and program managers to weigh the effectiveness of a proposed 


change with the resulting cost and readiness savings. — 
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IV. RESULTS 


A. OVERVIEW 


This chapter presents a database summary for annual flight operations by mishap 
class, and flight hours flown. Mishap causal frequencies are summarized for operations 
going back to 1986, and for the period of the study, 1993-98. Mishap coding results are 
presented by both number and frequency. All simulation model input parameters are then 
estimated and presented. Simulation results for the each mishap intervention strategy are 
presented for the existing baseline and for mishap frequency reductions of 10 percent, 30 
percent and 50 percent. The chapter concludes with a graphic comparison of cost and 


readiness results for all intervention categories. 


B. BACKGROUND UAV FLIGHT DATA 


Figure 7 is a graph of flight hours flown and the number of mishaps versus time. 
Table 3 contains a summary of the UAV annual flight hours, mishaps and associated rates 
for the period since the fielding of the Pioneer system by the Navy and Marine Corps in 
1986. In general, the annual flight hours flown is increasing and the mishap rate is 
decreasing. During Operation Desert Shield and Desert Storm, six UAV units flew just 
under 1000 flight hours. These units deployed with thirty air vehicles. At current mishap 
rates and during a deployment of equal duration, approximately one third of the air 
vehicles would be destroyed or damaged by flight mishaps. This prediction demonstrates 


unacceptable mission readiness and strains maintenance and repair capabilities. 
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UAV Mishap rate vs. Flight Hours 
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FIGURE 7: GRAPH OF MISHAP RATE AND ANNUAL FLIGHT HOURS 


Flight Mishap 
Year Class A Class B Class C Total Rate 
Hours 
(per 1000 hours) 


crs Ee ee le 
i ES ee a a ee ee ae 
SS lin a ea a a es 







a 
i a a a a a 
me eT 
a eta Pe | 7 |e __| ae 
(ee i a a ee 
ae a ee ae 
[EET eT il ll (RG a 2 
[i996 [soos] 9 | 9 | 5 | 23 | 1533 


er ene || 0 
et | ee | ee 
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TABLE 3: UAV MISHAPS BY YEAR AND CLASSIFICATION 
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&. MISHAP CODING 


The frequency of mishap cause occurrences is identified in Table 4. This data is 
consistent with the previous studies of Schmidt & Parker (1995) and Seagle (1997). 
Appendix C contains the complete mishap-coding database. Of note, the frequency of 
human factors related mishaps is not increasing. Rather, the reporting of human factors 
related mishaps is increasing in detail. The Mishap Investigative Reports (MIRs) 
submitted since FY 93 have increased the information available upon which to assign 
causal factors. Table 5 is a summary of each category of mishap causation. The 
simulation model evaluates seven mishap intervention strategies. Each intervention 


Strategy and its associated mishap classification category are identified in Table 6. 
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TABLE 4: MISHAP CAUSATION FREQUENCY 
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a aE. 86-98 | Fy 93-98 4 
CAUSALFACTOR | _ CODE | # FREQ) # (FREQ) 
Human Factors | CF | 857_(| 39.0% 55 [59.1% 
UnsafeAct_ | CA 85_|15.7%_ 35 | 37.6% 
po intended UAT | 6 | 7.2% 16 [17.2% 
po Mistake JUATM | | 4.9% 11 [11.8% 
P| CViolation ATV | 6 | 2.7% 6 | 6.5% 
po Unintended AU 19 | 8.5% _ 19 | 20.4% 
ee) a LSP WUAUSHLS | 5:87.13 szow 
eee ee apse ee A ae eee 
Unsafe Condition S| CC 87 | 16.6%] _37_ [39.8% 
| CCC“‘(‘!C*C Aeromedical” = JUCA_ | 9 | 4.0% 9 | 9.7% 
2 eRe PS wcerl) Mew)! 7ePemes0m 
JC Readiness Violation |UCV_ | 9 | 4.0% 9 | 9.7% 
Unsafe Supervision | US| 40 | 17.9% _40_[ 43.0% 
pL Unforeseen USU | 14 | 6.3% 14 [15.1% 
po Foreseen USF | | 4.9% 11 111.8% 
Electro-Mechanical | EM_|_ 158 170.9% 64 | 68.8% 
aa engine ENG 52_ | 23.3%) 23, Bag 
pd Electrical ELEC | 59 | 26.5% 20 [21.5% 
po Launcher failure LNCHR|_ 8 | 3.6% 2 | 2.2% 
pd Net recovery failure NET | 16 | 7.2% 7 | 7.5% 
ees | Sckware SOFT | 7 | 3212S ee 
| ther OTHER] _20_| 9.0% 8 _| 8.6% 
Unknown/ unspecified | SUNK | 8 | 3.6% _7_| 7.5% 


TABLE 5: MISHAP FREQUENCY BY CAUSATION CODE 
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Mishap Intervention Strategy Mishap Category 
Aeromedical Screening and Education Unsafe Conditions (UC) 
Aircrew Coordination Training / Crew a 
Unsafe Acts, Unsafe Conditions (UA/UC) 
Flight Simulator / TCS Unsafe Acts (UA) 
Automatic Takeoff and Landing System Unsafe Acts (UA) 
Personnel Stabilization Unsafe Acts, Unsafe Supervision (UA/US) 
Engine Upgrade Engine (ENG) 
Weatherizing the Air vehicle Electronic (ELEC) 


TABLE 6: MISHAP INTERVENTION STRATEGY AND ASSOCIATED CAUSAL CATEGORY 
D. STOCHASTIC MISHAP MODEL ESTIMATES 
1. Inter-event Times 


Figures 8 through 13 are histogram plots of the inter-arrival times for each 
partition of mishap causal factors. Overlaid on the chart is a rescaled probability density 
function (pdf) of the hypothesized exponential distribution. Table 7 summarizes the 
mean, standard deviation and rate parameters for the mishap category parameter 
estimates. Additionally, 95%, two-sided confidence intervals are presented for each rate 
parameter estimate. Note that the two confidence limits are not equidistant from the point 
estimate. This is due to the lack of symmetry in the exponential distribution. The 
Kolomogorov-Smirnov goodness of fit (KS g.o.f.) test for the exponential distribution is 
based on the estimated rate parameter. With a significance level (a) set at 0.05, the KS 


g.o.f. test fails to reject that any of the distributions are exponential. 
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FIGURE 8: HISTOGRAM PLOT OF UC DATA 
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FIGURE 9: HISTOGRAM PLOT OF UA/UC DATA 
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FIGURE | 1: HISTOGRAM PLOT OF UA/US DATA 
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FIGURE 13: HISTOGRAM PLOT OF ELEC DATA 
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TABLE 7: PARAMETER ESTIMATES FOR MEAN, STANDARD DEVIATION AND RATE 









2 Mishap Class Probability Parameters 


Table 8 is a summary of the calculations used to estimate the probability of each 
type of mishap (pa, pp, pc). Of note, the estimated probability of each class of mishap 
(Pa-Hat. PB-Hat> PC-Hat) Varies with the mishap category. For example, an engine failure is 
most likely to cause a catastrophic class A mishap, while the results of an unsafe act can 


be mitigated by other actions of the crew. 


a 
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TABLE 8: MISHAP CLASS PROBABILITIES 
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3. Cost Parameters 


Table 9 summarizes the parametric estimates for the mishap cost distributions. 
Each is hypothesized to be normally distributed. All cost figures are calculated as FY98 
dollars. Figures 14 through 16 are normal probability plots of the actual data versus the 
hypothesized distributions. Graphically, there are some discrepancies between the sample 
data and the hypothesized distribution. However, the KS g.o.f. tests the hypothesized 
distribution using the parameter estimates listed in Table 9. With a significance level (a) 


set at 0.05, the KS g.o.f. test fails to reject the normality of any of the cost distributions. 


TABLE 9: FY86-FY98 MISHAP CLASS COST DISTRIBUTION 
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FIGURE 14: NORMAL PROBABILITY PLOT OF CLASS A MISHAP COSTS 
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FIGURE 15: NORMAL PROBABILITY PLOT OF CLASS B MISHAP COSTS 
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FIGURE 16: NORMAL PROBABILITY PLOT OF CLASS C MISHAP COSTS 


4. Annual Flight Hours 


A linear regression of annual flight hours versus the natural log of fiscal year is 
performed on historic flight hours flown. The equation-projected estimate of flight hours 
for the next fiscal year (FY99) is 1,930 flight hours. This will be used as the number of 
annual flight hours in the simulation. Figure 17 contains a graph of the historic flight 


hours and the fitted equation, adjusted to time on a linear scale. 
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y = 651.77In(x) + 210.34 @ Actual FitHrs ——Fitted | 
x = FY87 thru FY98 : 


R? = 0.7191 























FIGURE 17: RESULTS OF ANNUAL FLIGHT HOUR REGRESSION 


E. STOCHASTIC MODEL SIMULATION 
1. Model: Baseline of total mishaps 


The aggregate mishap model is presented in Table 10. It is used as a baseline for 
comparison of the remaining models. Calculating the defined readiness factor, this model 
indicates that UAVs never achieve a mission ready condition. Also, UAV mishap costs 
typically exceed $10 million. The following model simulations are used to gain insight 
into the cost and mission readiness improvements over current baseline conditions made 


by targeting mishap causes with the specified strategies. 
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TABLE 10: AGGREGATE MISHAP MODEL 
2. Model: Increased Aeromedical Screening and Education 
Table 11 summarizes the unsafe conditions mishap intervention model. Even at 
current levels, the readiness indicator for unsafe conditions does not go below 80%. This 
is most likely the result of the relative probability of a class A mishaps being low 
compared to the aggregate model. As a result, costs are also kept low, accounting for 


approximately 1/3 of the aggregate mishap costs. 


Category: UC X-10% | X-30% | 1-50% 
Mean Cost (CY98 $M) $3.51 $3.09 $2.60 $1.94 


SD Cost (CY98 $M) $1.03 
% change in Cost Me -12.1% | 261% -44.8% 
Readiness Index 0.9970 
% change in Readiness | | 9.2% | 15.6% | 18.7% 





TABLE 11: UNSAFE CONDITIONS MODEL 
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3. Model: Aircrew Coordination Training / Crew Resource Management 


Table 12 summarizes the mishap intervention model for the aggregate of unsafe 
acts and unsafe conditions. ACT / CRM will have to reduce the mishap rate by over 10% 
in order to get readiness above the 80% level. Additionally, at $4 million per year in 
mishap cost contributions, unsafe acts and unsafe conditions combined contribute to 


approximately one-third of mishap costs. 


Sr OOF 


TABLE 12: AGGREGATE UNSAFE ACTS / UNSAFE CONDITIONS MODEL 



















4. Model: UAV Flight Simulator / Automated Take off and Landing 
System 


Table 13 summarizes the mishap intervention model for unsafe acts. This model 
is used to evaluate the potential results of both the flight simulator and of the take off and 
landing aids. While the category of unsafe acts contributes to 37% percent of mishaps, 
their effect on readiness is not as profound. Even in the baseline case, unsafe acts have a 
ready index of nearly 90 percent. Because of their nature, unsafe acts will contribute to 
mishap occurrences, but their individual effects do not have as great an impact on overall 


changes in readiness and costs. 
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TABLE 13: UNSAFE ACTS MODEL 
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Si Model: Personnel Stabilization 


Table 14 summarizes the mishap intervention model for the aggregate of unsafe 
acts and unsafe supervision. This is the only model that incorporates the effects of unsafe 
supervision. Although difficult to isolate, unsafe supervision when coupled with unsafe 
acts does have a profound effect on both cost and readiness. The model bears out that 


personnel stability has the potential to significantly improved readiness and reduced cost. 


Category: UA/US A-10% | 01-30% | A-50% 
Mean Cost (CY98 $M) $4.52 $4.20 $3.39 


SD Cost (CY98 $M) $1.76 | $1.60 


Readiness Index 0.5570 0.6760 0.8710 0.9750 
% change in Readiness Y Tak 21.4% 56.4% 75.0% 


TABLE 14: AGGREGATE ACT/UNSAFE SUPERVISION MODEL 
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6. Model: Engine Replacement 

Table 15 summarizes the mishap intervention model for engine failure related 
events. Engine failure accounts for nearly 25 percent of mishaps. Intuitively, 
modifications to the engine should cause a significant cost reduction and readiness 
improvement. However, the model results indicate that the effect on mission readiness is 
negligible since the baseline readiness factor already exceeds 95 percent. Additionally, 


the budgetary cost of these mishaps is less than or equal to other mishap categories. 


% change in Cost —- Tl] 762 | 2802 | a0 | 


TABLE 15: ENGINE MODEL 












% change in Readiness 





is Model: Electronic Waterproofing 


Table 16 summarizes the mishap intervention model for the improvements to 
electronic component reliability. Electronic component failure is cited in over 20% of 
UAV mishaps, but similar to engine failure, its impact on readiness and annual costs is 
overshadowed by the other causal categories. The model indicates that the effect of 


electrical failures upon mission readiness is minimal. 
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TABLE 16: ELECTRONIC MODEL 
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He MODEL COMPARISON 


Figures 18 and 19 are a comparison between the results of the mishap intervention 
simulations. Figure 18 compares the cost reduction in each category for the four runs of 
the model. Clearly, there is improvement across all strategies, but improvements in 
UA/UC and UA/US appear to cause more significant cost savings. Figure 19 illustrates 
the mission readiness index improvement. Inspection of the graph reveals that addressing 
the UC, UA/UC and UA/US categories will have the greatest impact on mission 
readiness. According to the simulation, none of other causal factors reduces the readiness 


factor below 80 percent. 
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FIGURE 18: MISHAP COST REDUCTION 
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FIGURE 19: MISHAP READINESS INDEX IMPROVEMENT 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. MISHAP CLASSIFICATION 


Unmanned Aerial Vehicle (UAV) mishaps are successfully categorized by the 
Human Factors Accident and Classification System (HFACS) using the Naval Aviation 
Safety Program (OPNAVINST 3750.6Q) mishap report procedures. However, 
granularity at the lowest tiers of the HFACS model can be difficult to ascertain. The 
information available for analysis appears to vary with the writer of the report. If that 
individual is aware and comfortable with HFACS, the report tends to address all of the 
details required for future analysis. Omen ice reports contain generalities and lack the 
specific details necessary to conduct an in depth analysis of the data. The planned 
incorporation of the HFACS taxonomy into the next revision of OPNAVINST 3750.6R, 
should standardize reporting procedures, and educate those preparing mishap reports on 
the scope that human factors can have on flight operations. 

This analysis of mishap classification is conducted independently of the data 
partition of Schmidt & Parker (1995) and Seagle (1997). The three separate mishap 
categorizations are similar in terms of the relative frequency and causes of mishaps. 
Individual judgments were required by those conducting the study in order to place a 
particular mishap into a causation category. While not always agreeing in the exact 
classification, the three studies do conclude that human error is at least partially 
attributable for approximately one half of UAV mishaps. Engine and electronic failure 


each account for 20 to 25 percent of mishaps. Finally, although the mishap rate is 
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decreasing, Current mishap damage and losses will continue to have a profound affect on 


mission readiness. 


B. MISHAP MODELING 


The mishap occurrence distribution is modeled effectively as a Poisson process. 
Cost data and mishap class data distributions can also be modeled by specific 
distributions. These distributional quantities are effectively input into the stochastic 


model, allowing the analyst to simulate annual UAV flight operations accurately. 


Cc. MODEL RESULTS 


The aggregate UAV mishap rate must be reduced by one half in order to achieve a 
significant change in total mishap occurrences. This mishap reduction still only raises the 
readiness index to 0.300. (See Figure 19.) In order to focus the mishap intervention 
Strategies, they must be analyzed in isolation. Under the current baseline rates two 
mishap categories, Unsafe Acts and Unsafe Conditions (UA/UC), and Unsafe Acts and 
Unsafe Supervision (UA/US), cause the readiness index to fall below 0.800. The 
simulations demonstrate that improvements up to 40.8 percent and 75.0 percent, 
respectively, can be achieved in these categories. This indicates that they should be 
considered primary targets for intervention Strategies. 

The other impact of the simulated UAV mishaps is cost. Again, the UA/UC and 
UA/US categories are the two most costly mishap causal factors. Each contributes 
greater than $4 million (FY98$) to annual mishap costs in the model. Intervention in 


these categories can reduce costs by 44.4 percent and 40.8 percent, respectively. 
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Although not considered in this model, intervention strategies should be compared to find 
the best value for the money invested. 

Contrary to expectations, incorporating engineering modifications, (engine 
improvement/replacement, and electronic waterproofing) will have marginal effects on 
readiness and cost. At current mishap rates, the engine and electronic configuration do 
not degrade mission readiness below 95 percent. Additionally, their mishap costs are 
approximately the same as the other mishap categories. The cost involved in research, 
development and procurement will most likely exceed current mishap cost predictions 


and can be better spent improving other aspects of the Pioneer system. 


D. RECOMMENDATIONS 


The evaluation of the stochastic model results points to mishap intervention 
measures for the UA/UC and UA/US categories. This research proposes improvements 
to aircrew coordination training and crew resource management in order to alleviate the 
effects of the UA/UC category. Initiatives in this area will not only improve Pioneer 
UAV operations, but will have a listing impact on the UAV community, regardless of the 
system employed. When the Pioneer is eventually replaced, only minor modifications 
should be necessary to adjust to the idiosyncrasies of the new systems. The second area 
of intervention recommended is UAV personnel stabilization. The UAV community is 
still relatively new to Naval Aviation operations. As such, the community needs to 
mature. Unit leaders, both officer and enlisted, should have experience and knowledge of 
the system in order to effectively manage unit operations and individual crewmembers. A 


UAV career path should be created to track these individuals and assign them 


ga 


appropriately throughout their careers so that the UAV community can benefit from the 


stability and "corporate knowledge” enjoyed by other Naval flight communities. 
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APPENDIX A: PIONEER UAV SYSTEM DESCRIPTION 


A. SYSTEM COMPONENTS 


A Pioneer UAV system consists of five Pioneer air vehicles, a ground control 
station (GCS), a portable control station (PCS), a tracking communications unit (TCU), a 
data link, two remote receiver stations (RRS) and a reconnaissance payload. The system 
can be operated aboard specially configured USS Austin Class Landing Platform Dock 
(LPD-4) ships or from prepared airstrips ashore. 


1. The Air Vehicle 





FIGURE 20: PIONEER AIR VEHICLE 


The Pioneer vehicle air is 14 feet long and is pusher-propeller driven, powered by 
a 26 hp, two stroke, twin cylinder, rear mounted engine, similar to a snowmobile engine. 
The air vehicle is made of fiberglass, Kevlar and other low cost composite materials, and 
weighs 463 lbs. The air vehicle can operate up to an altitude of 15,000 feet, but normally 
flies between 3000 and 5000 feet in order to optimize payload performance. Because the 
air vehicle uses a laminated wood propeller, and the electronic components are not 


weatherized, the UAV cannot fly through visible moisture (fog, clouds, rain, etc.) or icing 
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conditions. The air vehicle has up to four-hour time of flight at a cruise speed of 65 
knots, which normally translates into 2-2.5 hours in an objective area depending on the 
proximity of the airstrip to the objective. 

The air vehicle is launched using one of three methods. The rocket-assisted take 
off (RATO) is the only method available for shipboard operations. A rocket is placed 
under the vehicle to propel it into the air. Having reached sufficient altitude and airspeed, 
the rocket motor shuts down and is jettisoned from the UAV. Land-based units can also 
conduct RATO launch. Additionally on land, the UAV can use a standard rolling take of 
from a 1500-foot runway. Because of restrictive crosswind parameters, or air density 
constraints, a rolling takeoff may not be possible. For these instances, a pneumatic 
launcher mounted on a 5-ton truck can propel the vehicle to the minimum altitude and 


airspeed to transition to vehicle-powered flight. 





FIGURE 21: RATO TAKEOFF 
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There are two ways to safely recover the air vehicle. Operations at sea require the 
UAV to be flown into a large net suspended across the aft part of a ship’s helicopter flight 
deck. Once, in the net, the recovery system collapses around the UAV allowing it to be 
lifted out. Because this recovery method is tantamount to a controlled crash, there is 
frequent damage to the UAV. The second recovery method, need ashore, iS an arrested 
recovery by a miniature tailhook on an airstrip. While much more suitable for a crash 
free recovery, cross wind limitation must be monitored in order to assure a successful 


recovery (MAWTS-1, 1997). 





FIGURE 23: UAV SHIPBOARD LANDING 
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2: The Ground Control Station (GCS) 


The GCS is the focus of activity for UAV missions. The system can either be 
land based or installed aboard ship. The GCS consists of three electronics bays manned 
by two operators. The pilot bay includes all controls, instruments and displays required to 
safely "fly" the vehicle. The observer bay provides control and display of the imaging 
payloads carried by the UAV. The tracking bay displays the UAV position based on data 


from the TCU and global positioning satellites (GPS) (Pioneer UAV, INC., 1999). 





FIGURE 24: INSIDE THE GCS 


S The Portable Control Station (PCS) 


The PCS provides the capability to control the UAV during pre-flight, launch and 
recovery operations, allowing the GCS to locate where it can most effectively conduct the 
mission. Because the air vehicle relies on line of site communications between the 


control station and the air vehicle, split sight operations are common in rugged, 
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compartmented terrain. The PCS provides the ability to control the launch sequence from 
a local airstrip, and then steer the air vehicle to a predetermined handover point. There, 
the GCS, operating from a more advantageous location, can take control of the UAV and 


conduct the mission further down range (MAWTS-1, 1997). 





FIGURE 25: INSIDE THE PCS CONTROL BAY 
4. The Tracking Control Unit (TCU) 


The TCU shelter contains the UAV communication equipment and antennas. The 
TCU contains a sophisticated, jam resistant, C-band, 100 nmi. range data link. Both the 
video and telemetry link use directional antennas between the air vehicle and the TCU in 
order to ensure video quality and minimize the probability of data link intercept by the 
enemy. The system also has an omni-directional, UHF backup link for redundancy. The 
TCU can be remoted 1000 meters from the GCS by fiber-optic cable, enhancing the 


system’s and personnel battlefield survivability (Pioneer UAV, INC., 1999). 
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FIGURE 26: THE TCU 
5. Remote Receiving Station (RRS) 


The ruggedized RRS provides real-time reception of the UAV video picture at 
remote locations. The Marine Corps has mounted the RRS on a high mobility multi- 
wheeled vehicle (HMMWV), a light armored vehicle (LAV) and aboard a UH-1IN Huey 
helicopter, allowing the tactical commander to have real-time imagery regardless of 


where the command post is locating (MAWTS-1, 1997). 





FIGURE 27: THE RRS 
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6. Reconnaissance Payloads 


The air vehicle can carry one of two separate, gyrostabilized payloads: the MKD- 
200 electro-optical day camera, and the MKD-400(C) forward-looking infrared (FLIR) 
night camera. The MKD-200 E-O camera can detect targets up to 18 km range, and 
recognizes targets at 3 km range. The MKD-400(C) FLIR camera can detect a target at 8 
km range, and recognizes a target at up to 4 km. Camera performance is enhanced by 
increased thermal differential between the target and the surrounding background 


(MAWTS-1, 1997). 





FIGURE 28: UAV PAYLOADS 


B. CREW COMPOSITION 


The term "unmanned" is actually a misnomer when applied to the Unmanned 
Aerial Vehicle system because UAV operations involve many participants. The essential 
members of a UAV crew include a Mission Commander, an Internal Pilot, a Payload 


Operator and an External Pilot. Additionally launch and recovery teams and maintenance 
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personnel will be involved in flight operations. The responsibilities of each crewmember 
are summarized below (JUAVTOPS, 1997): 


I. Mission Commander (MC) 


The MC is typically a rated Naval Aviator or Naval Flight Officer, who has the 
supervisory responsibility for the UAV mission. This includes organizing the entire flight 
crew, coordination with external agencies and supported units during pre-mission 
planning, execution, and post mission debriefing. 


Jap Internal Pilot (IP) 


Typically a senior enlisted aviation rating, who flies the UAV down range, 
monitors instruments to ensure proper operation, and assists the payload operator (PO) to 
get optimal camera position. The IP is also responsible for in-flight emergencies. 


3: Payload Operator (PO) 


The PO is an enlisted operator who controls the UAV camera and monitors the 
tracker bay to insure proper orientation. The PO assists the IP through visual navigation 
and during in-flight emergencies. 


4. External Pilot (EP) 


The EP, typically an enlisted operator, flies the UAV during launch and recovery 
operations. He coordinates the UAV handoff to the IP, and handles all launch and 
recovery emergencies. 


5. Other Crewmembers 


Depending on the complexity of the mission, and the experience of the crew, 
additional personnel may be required to augment the basic crew. Intelligence personnel 


may be involved to exploit the video imagery and pass that information on to the 
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appropriate supported units. If the mission calls for fire support adjustment, an artillery 
or naval gunfire forward observer will be added to the crew. The crew is then rounded 


out with UAV maintenance and communications personnel. 
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APPENDIX B: MISHAP SIMULATION CODE 


MishapSim( ) 


function(runs, lamda, PHat, fltHours) 


{ 


+H + # # 


HE HE HEHEHE HEHEHE HEHEHE HEHEHE HEHE HEHEHE HEHE HIE EH EE HEHE 


TnpuG: 

runs is the number of repetitions for the simulation 
lamda is the inter-arrival mishap rate for an exponential 
Gi stributien. 

PHat is a vector of probabilities for a class A, B orc 
mishap as calculated by the ClassSim( ) function. 
fltHours is the length of the flight period to be 


Simulated. 


Function: 

This simulation 15 basedeen ameanntiet flight hour period: 
A random exponential variable with the inputted rate 
parameter is used to simulate the occurrence of a mishap. 


This is used to increment the time. 


Once a mishap is generated, a second random sample is 
drawn to determine the type of mishap (class A, B or C). 
The damage cost is also determined by the cost 


distribution data that is hardwired into the program. 


A cost vector and readiness index vector are built as 
each annual flight period is completed. The weighted 
readiness index weighs a class A = 3, class B = 2, class 
C =1. The maximum index in order to be "MISSION READY" 
is 21. A readiness index greater than 21 indicates "NOT 
MESSTON READY: 
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# Returns: 
# The Function returns a vector of simulated annual costs, 
# the mean and standard deviation of the mishap cost and 
# the average readiness index for the year. 
HHt HH 

readiness <- vector("integer", runs) 

totalCost <- vector("double", runs) 

home: 21, > runs) ost 


time <- 0 


Meshapclass <= c('A' yea". "C") 
cost <- 0 
numA <- 0 
rggbeqgls; =I 8) 


Hume j= — 50 
while(time < fltHours) { 
time <- time + rexp(1, lamda/1000) 
mishapType <- sample(mishapClass, 1, replace 
= 7, PHat) 
mamichapiype == "“Al) { 
damage <- rnorm(l, 812000, 187000) 
numA <- numA + 1 
} 


Le(mMacnaotype =— "B’) { 
damage <- max(200000, rnorm(1, 480000, 
earaO0'O )*) 
numB <- numB + 1 
} 
i1f£(mishapType == "C") { 
damage <- max(10000, rnorm(l1, 88000, 
64000) ) 
numC <- numc + 1 
} 


cost <- cost + damage 


} 


totalGesetef[i)] <= cost 
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readiness[i] <- l 
readyFactor <- 3 * numA + 2 * numB + numCc 
LE (readyvERacter >= Zim 4 


readiness[i] <- 0 


} 


mishapCost <- mean(totalCost) 

SDCost <- sqrt(var(totalCost) ) 

readyIndex <- mean(readiness) 

return(totalCost, mishapCost, SDCost, readyIndex) 
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